Emergent neural dynamics and geometry for generalization in a transitive inference task.

Relational cognition-the ability to infer relationships that generalize to novel combinations of objects-is fundamental to human and animal intelligence. Despite this importance, it remains unclear how relational cognition is implemented in the brain due in part to a lack of hypotheses and predictions at the levels of collective neural activity and behavior. Here we discovered, analyzed, and experimentally tested neural networks (NNs) that perform transitive inference (TI), a classic relational task (if A > B and B > C, then A > C). We found NNs that (i) generalized perfectly, despite lacking overt transitive structure prior to training, (ii) generalized when the task required working memory (WM), a capacity thought to be essential to inference in the brain, (iii) emergently expressed behaviors long observed in living subjects, in addition to a novel order-dependent behavior, and (iv) expressed different task solutions yielding alternative behavioral and neural predictions. Further, in a large-scale experiment, we found that human subjects performing WM-based TI showed behavior inconsistent with a class of NNs that characteristically expressed an intuitive task solution. These findings provide neural insights into a classical relational ability, with wider implications for how the brain realizes relational cognition.

A mathematical theory of relational generalization in transitive inference.

Humans and animals routinely infer relations between different items or events and generalize these relations to novel combinations of items ("compositional generalization"). This allows them to respond appropriately to radically novel circumstances and is fundamental to advanced cognition. However, how learning systems (including the brain) can implement the necessary inductive biases has been unclear. Here we investigated transitive inference (TI), a classic relational task paradigm in which subjects must learn a relation (A > B and B > C) and generalize it to new combinations of items (A > C). Through mathematical analysis, we found that a broad range of biologically relevant learning models (e.g. gradient flow or ridge regression) perform TI successfully and recapitulate signature behavioral patterns long observed in living subjects. First, we found that models with item-wise additive representations automatically encode transitive relations. Second, for more general representations, a single scalar "conjunctivity factor" determines model behavior on TI and, further, the principle of norm minimization (a standard statistical inductive bias) enables models with fixed, partly conjunctive representations to generalize transitively. Finally, neural networks in the "rich regime," which enables representation learning and often leads to better generalization, deviate in task behavior from living subjects and can make generalization errors. Our findings show systematically how minimal statistical learning principles can explain the rich behaviors empirically observed in TI in living subjects, uncover the mechanistic basis of transitive generalization in standard learning models, and lay out a formally tractable approach to understanding the neural basis of relational generalization.

Alteration of functional connectivity in the cortex and major brain networks of non-human primates following focused ultrasound exposure in the dorsal striatum.

BACKGROUND: Focused ultrasound (FUS) is a non-invasive neuromodulation technology that is being investigated for potential treatment of neurological and psychiatric disorders. FUS combined with microbubbles can temporarily open the intact blood-brain barrier (BBB) of animals and humans, and facilitate drug delivery. FUS exposure, either with or without microbubbles, has been demonstrated to alter the behavior of non-human primates (NHP), and previous studies have demonstrated the transient and long-term effects of FUS neuromodulation on functional connectivity using resting state functional MRI. The behavioral effects of FUS vary depending on whether or not it is applied in conjunction with microbubbles to open the BBB, but it is unknown whether opening the BBB affects functional connectivity differently than FUS alone. OBJECTIVE: To compare the effects of applying FUS alone (FUS neuromodulation) and FUS with microbubbles (FUS-BBB opening) on changes of resting state functional connectivity in NHP. METHODS: We applied 2 min FUS exposure without (neuromodulation) and with microbubbles (BBB opening) in the dorsal striatum of lightly anesthetized non-human primates, and acquired resting state functional MRI 40 min respectively after FUS exposure. The functional connectivity (FC) in the cortex and major brain networks between the two approaches were measured and compared. RESULTS: When applying FUS exposure to the caudate nucleus of NHP, we found that both FUS neuromodulation can activate FC between caudate and insular cortex, while inhibiting the FC between caudate and motor cortex. FUS-BBB opening can activate FC between the caudate and medial prefrontal cortex, and within the frontotemporal network (FTN). We also found both FUS and FUS-BBB opening can significantly activate FC within the default mode network (DMN). CONCLUSION: The results suggest applying FUS to a deep brain structure can alter functional connectivity in the DMN and FTN, and that FUS neuromodulation and FUS-mediated BBB opening can have different effects on patterns of functional connectivity.

Superstitious learning of abstract order from random reinforcement.

Humans and other animals often infer spurious associations among unrelated events. However, such superstitious learning is usually accounted for by conditioned associations, raising the question of whether an animal could develop more complex cognitive structures independent of reinforcement. Here, we tasked monkeys with discovering the serial order of two pictorial sets: a "learnable" set in which the stimuli were implicitly ordered and monkeys were rewarded for choosing the higher-rank stimulus and an "unlearnable" set in which stimuli were unordered and feedback was random regardless of the choice. We replicated prior results that monkeys reliably learned the implicit order of the learnable set. Surprisingly, the monkeys behaved as though some ordering also existed in the unlearnable set, showing consistent choice preference that transferred to novel untrained pairs in this set, even under a preference-discouraging reward schedule that gave rewards more frequently to the stimulus that was selected less often. In simulations, a model-free reinforcement learning algorithm (Q-learning) displayed a degree of consistent ordering among the unlearnable set but, unlike the monkeys, failed to do so under the preference-discouraging reward schedule. Our results suggest that monkeys infer abstract structures from objectively random events using heuristics that extend beyond stimulus-outcome conditional learning to more cognitive model-based learning mechanisms.

Influence of Rule- and Reward-based Strategies on Inferences of Serial Order by Monkeys.

Knowledge of transitive relationships between items can contribute to learning the order of a set of stimuli from pairwise comparisons. However, cognitive mechanisms of transitive inferences based on rank order remain unclear, as are relative contributions of reward associations and rule-based inference. To explore these issues, we created a conflict between rule- and reward-based learning during a serial ordering task. Rhesus macaques learned two lists, each containing five stimuli that were trained exclusively with adjacent pairs. Selection of the higher-ranked item resulted in rewards. "Small reward" lists yielded two drops of fluid reward, whereas "large reward" lists yielded five drops. Following training of adjacent pairs, monkeys were tested on novels pairs. One item was selected from each list, such that a ranking rule could conflict with preferences for large rewards. Differences between the corresponding reward magnitudes had a strong influence on accuracy, but we also observed a symbolic distance effect. That provided evidence of a rule-based influence on decisions. RT comparisons suggested a conflict between rule- and reward-based processes. We conclude that performance reflects the contributions of two strategies and that a model-based strategy is employed in the face of a strong countervailing reward incentive.

Positional inference in rhesus macaques.

Understanding how organisms make transitive inferences is critical to understanding their general ability to learn serial relationships. In this context, transitive inference (TI) can be understood as a specific heuristic that applies broadly to many different serial learning tasks, which have been the focus of hundreds of studies involving dozens of species. In the present study, monkeys learned the order of 7-item lists of photographic stimuli by trial and error, and were then tested on "derived" lists. These derived test lists combined stimuli from multiple training lists in ambiguous ways, sometimes changing their order relative to training. We found that subjects displayed strong preferences when presented with novel test pairs, even when those pairs were drawn from different training lists. These preferences were helpful when test pairs had an ordering congruent with their ranks during training, but yielded consistently below-chance performance when pairs had an incongruent order relative to training. This behavior can be explained by the joint contributions of transitive inference and another heuristic that we refer to as "positional inference." Positional inferences play a complementary role to transitive inferences in facilitating choices between novel pairs of stimuli. The theoretical framework that best explains both transitive and positional inferences is a spatial model that represents both the position of each stimulus and its uncertainty. A computational implementation of this framework yields accurate predictions about both correct responses and errors on derived lists.

Transitive inference after minimal training in rhesus macaques (Macaca mulatta).

Rhesus macaques, when trained for several hundred trials on adjacent items in an ordered list (e.g., A > B, B > C, C > D), are able to make accurate transitive inferences (TI) about previously untrained pairs (e.g., A > C, B > D). How that learning unfolds during training, however, is not well understood. We sought to measure the relationship between the amount of TI training and the resulting response accuracy in 4 rhesus macaques using seven-item lists. The training conditions included the absolute minimal case of presenting each of the six adjacent pairs only once prior to testing. We also tested transfer to nonadjacent pairs with 24 and 114 training trials. Because performance during and after small amounts of training is expected to be near chance levels, we developed a descriptive statistical model to estimate potentially subtle learning effects in the presence of much larger random response variability and systematic bias. These results suggest that subjects learned serial order in an incremental fashion. Thus, rather than performing transitive inference by a logical process, serial learning in rhesus macaques proceeds in a manner more akin to a statistical inference, with an initial uncertainty about list position that gradually becomes more accurate as evidence accumulates. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Safety evaluation of a clinical focused ultrasound system for neuronavigation guided blood-brain barrier opening in non-human primates.

An emerging approach with potential in improving the treatment of neurodegenerative diseases and brain tumors is the use of focused ultrasound (FUS) to bypass the blood-brain barrier (BBB) in a non-invasive and localized manner. A large body of pre-clinical work has paved the way for the gradual clinical implementation of FUS-induced BBB opening. Even though the safety profile of FUS treatments in rodents has been extensively studied, the histological and behavioral effects of clinically relevant BBB opening in large animals are relatively understudied. Here, we examine the histological and behavioral safety profile following localized BBB opening in non-human primates (NHPs), using a neuronavigation-guided clinical system prototype. We show that FUS treatment triggers a short-lived immune response within the targeted region without exacerbating the touch accuracy or reaction time in visual-motor cognitive tasks. Our experiments were designed using a multiple-case-study approach, in order to maximize the acquired data and support translation of the FUS system into human studies. Four NHPs underwent a single session of FUS-mediated BBB opening in the prefrontal cortex. Two NHPs were treated bilaterally at different pressures, sacrificed on day 2 and 18 post-FUS, respectively, and their brains were histologically processed. In separate experiments, two NHPs that were earlier trained in a behavioral task were exposed to FUS unilaterally, and their performance was tracked for at least 3 weeks after BBB opening. An increased microglia density around blood vessels was detected on day 2, but was resolved by day 18. We also detected signs of enhanced immature neuron presence within areas that underwent BBB opening, compared to regions with an intact BBB, confirming previous rodent studies. Logistic regression analysis showed that the NHP cognitive performance did not deteriorate following BBB opening. These preliminary results demonstrate that neuronavigation-guided FUS with a single-element transducer is a non-invasive method capable of reversibly opening the BBB, without substantial histological or behavioral impact in an animal model closely resembling humans. Future work should confirm the observations of this multiple-case-study work across animals, species and tasks.

Negative allosteric modulation of metabotropic glutamate receptor 5 attenuates alcohol self-administration in baboons.

Many of the behavioral symptoms that define alcohol use disorder (AUD) are thought to be mediated by amplified glutamatergic activity. As a result, previous preclinical studies have investigated glutamate receptor inhibition as a potential pharmacotherapy for AUD, particularly the metabotropic glutamate receptor 5 (mGlu5). In rodents, mGlu5 negative allosteric modulators (NAMs) have been shown to decrease alcohol self-administration. However, their effect on non-human primates has not previously been explored. To bridge this gap, the effects of mGlu5 NAM pretreatment on sweetened alcohol (8% w/v in diluted KoolAid) self-administration in female baboons were evaluated. Two different mGlu5 NAMs were tested: 1) 3-2((-Methyl-4-thiazolyl) ethynyl) pyridine (MTEP) which was administered at a dose of 2 mg/kg IM; and 2) auglurant (N-(5-fluoropyridin-2-yl)-6-methyl-4-(pyrimidin-5-yloxy)picolinamide), a newly developed NAM, which was tested under two different routes (0.001, 0.01, 0.03, 0.1 mg/kg IM and 0.1, 0.3, 1.0 mg/kg PO). MTEP decreased both fixed ratio and progressive ratio responding for sweetened alcohol. Auglurant, administered IM, decreased alcohol self-administration at doses that did not affect self-administration of an alcohol-free sweet liquid reward (0.01 to 0.1 mg/kg). Oral administration of auglurant was not effective in decreasing alcohol self-administration. Our results extend positive findings from rodent studies on mGlu5 regulation of alcohol drinking to female baboons and further strengthen the rationale for targeting mGlu5 in clinical trials for AUD.

Combining brain perturbation and neuroimaging in non-human primates.

Brain perturbation studies allow detailed causal inferences of behavioral and neural processes. Because the combination of brain perturbation methods and neural measurement techniques is inherently challenging, research in humans has predominantly focused on non-invasive, indirect brain perturbations, or neurological lesion studies. Non-human primates have been indispensable as a neurobiological system that is highly similar to humans while simultaneously being more experimentally tractable, allowing visualization of the functional and structural impact of systematic brain perturbation. This review considers the state of the art in non-human primate brain perturbation with a focus on approaches that can be combined with neuroimaging. We consider both non-reversible (lesions) and reversible or temporary perturbations such as electrical, pharmacological, optical, optogenetic, chemogenetic, pathway-selective, and ultrasound based interference methods. Method-specific considerations from the research and development community are offered to facilitate research in this field and support further innovations. We conclude by identifying novel avenues for further research and innovation and by highlighting the clinical translational potential of the methods.

Category learning in a transitive inference paradigm.

The implied order of a ranked set of visual images can be learned without reliance on information that explicitly signals their order. Such learning is difficult to explain by associative mechanisms, but can be accounted for by cognitive representations and processes such as transitive inference. Our study sought to determine if those processes also apply to learning categories of images. We asked whether participants can (a) infer that stimulus images belonged to familiar categories, even when the images for each trial were unique, and (b) sort those categories into an ordering that obeys transitivity. Participants received minimal verbal instruction and a single session of training. Despite this, they learned the implied order of lists of fixed stimuli and lists of ordered categories, using trial-unique exemplars. We trained two groups, one for which stimuli were constant throughout training and testing (n = 60), and one for which exemplars of each category were trial-unique (n = 50). Our findings suggest that differing cognitive processes may underpin serial learning when learning about specific stimuli as opposed to stimulus categories.

Learned Representation of Implied Serial Order in Posterior Parietal Cortex.

Monkeys can learn the implied ranking of pairs of images drawn from an ordered set, despite never seeing all of the images simultaneously and without explicit spatial or temporal cues. We recorded the activity of posterior parietal cortex (including lateral intraparietal area LIP) neurons while monkeys learned 7-item transitive inference (TI) lists with 2 items presented on each trial. Behavior and neuronal activity were significantly influenced by the ordinal relationship of the stimulus pairs, specifically symbolic distance (the difference in rank) and joint rank (the sum of the ranks). Symbolic distance strongly predicted decision accuracy and learning rate. An effect of joint rank on performance was found nested within the symbolic distance effect. Across the population of neurons, there was significant modulation of firing correlated with the relative ranks of the two stimuli presented on each trial. Neurons exhibited selectivity for stimulus rank during learning, but not before or after. The observed behavior is poorly explained by associative or reward mechanisms, and appears more consistent with a mental workspace model in which implied serial order is mapped within a spatial framework. The neural data suggest that posterior parietal cortex supports serial learning by representing information about the ordinal relationship of the stimuli presented during a given trial.

Absolute and relative knowledge of ordinal position on implied lists.

Does serial learning result in specific associations between pairs of items, or does it result in a cognitive map based on relations of all items? In 2 experiments, we trained human participants to learn various lists of photographic images. We then tested the participants on new lists of photographic images. These new lists were constructed by selecting only 1 image from each list learned during training. In Experiment 1, participants were trained to choose the earlier (experimenter defined) item when presented with adjacent pairs of items on each of 5 different 5-item lists. Participants were then tested on derived lists, in which each item retained its original ordinal position, even though each of the presented pairs was novel. Participants performed above chance on all of the derived lists. In Experiment 2, a different group of participants received the same training as those of Experiment 1, but the ordinal positions of items were systematically changed on each derived list. The response accuracy for Experiment 2 varied inversely with the degree to which an item's original ordinal position was changed. These results can be explained by a model in which participants learned to make both positional inferences about the absolute rank of each stimulus, and transitive inferences about the relative ranks of pairs of stimuli. These inferences enhanced response accuracy when ordinal position was maintained, but not when it was changed. Our results demonstrate quantitatively that, in addition to item-item associations that participants acquire while learning a list of arbitrary items, they form a cognitive map that represents both experienced and inferred relationships. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Discovering Implied Serial Order Through Model-Free and Model-Based Learning.

Humans and animals can learn to order a list of items without relying on explicit spatial or temporal cues. To do so, they appear to make use of transitivity, a property of all ordered sets. Here, we summarize relevant research on the transitive inference (TI) paradigm and its relationship to learning the underlying order of an arbitrary set of items. We compare six computational models of TI performance, three of which are model-free (Q-learning, Value Transfer, and REMERGE) and three of which are model-based (RL-Elo, Sequential Monte Carlo, and Betasort). Our goal is to assess the ability of these models to produce empirically observed features of TI behavior. Model-based approaches perform better under a wider range of scenarios, but no single model explains the full scope of behaviors reported in the TI literature.

Reward associations do not explain transitive inference performance in monkeys.

Most accounts of behavior in nonhuman animals assume that they make choices to maximize expected reward value. However, model-free reinforcement learning based on reward associations cannot account for choice behavior in transitive inference paradigms. We manipulated the amount of reward associated with each item of an ordered list, so that maximizing expected reward value was always in conflict with decision rules based on the implicit list order. Under such a schedule, model-free reinforcement algorithms cannot achieve high levels of accuracy, even after extensive training. Monkeys nevertheless learned to make correct rule-based choices. These results show that monkeys' performance in transitive inference paradigms is not driven solely by expected reward and that appropriate inferences are made despite discordant reward incentives. We show that their choices can be explained by an abstract, model-based representation of list order, and we provide a method for inferring the contents of such representations from observed data.

Modulation of Brain Function and Behavior by Focused Ultrasound.

PURPOSE OF REVIEW: The past decade has seen rapid growth in the application of focused ultrasound (FUS) as a tool for basic neuroscience research and potential treatment of brain disorders. Here, we review recent developments in our understanding of how FUS can alter brain activity, perception and behavior when applied to the central nervous system, either alone or in combination with circulating agents. RECENT FINDINGS: Focused ultrasound in the central nervous system can directly excite or inhibit neuronal activity, as well as affect perception and behavior. Combining FUS with intravenous microbubbles to open the blood-brain barrier also affects neural activity and behavior, and the effects may be more sustained than FUS alone. Opening the BBB also allows delivery of drugs that do not cross the intact BBB including viral vectors for gene delivery. SUMMARY: While further research is needed to elucidate the biophysical mechanisms, focused ultrasound, alone or in combination with other factors, is rapidly maturing as an effective technology for altering brain activity. Future challenges include refining control over targeting specificity, the volume of affected tissue, cell-type specificity (excitatory or inhibitory), and the duration of neural and behavioral effects.

Efficient Blood-Brain Barrier Opening in Primates with Neuronavigation-Guided Ultrasound and Real-Time Acoustic Mapping.

Brain diseases including neurological disorders and tumors remain under treated due to the challenge to access the brain, and blood-brain barrier (BBB) restricting drug delivery which, also profoundly limits the development of pharmacological treatment. Focused ultrasound (FUS) with microbubbles is the sole method to open the BBB noninvasively, locally, and transiently and facilitate drug delivery, while translation to the clinic is challenging due to long procedure, targeting limitations, or invasiveness of current systems. In order to provide rapid, flexible yet precise applications, we have designed a noninvasive FUS and monitoring system with the protocol tested in monkeys (from in silico preplanning and simulation, real-time targeting and acoustic mapping, to post-treatment assessment). With a short procedure (30 min) similar to current clinical imaging duration or radiation therapy, the achieved targeting (both cerebral cortex and subcortical structures) and monitoring accuracy was close to the predicted 2-mm lower limit. This system would enable rapid clinical transcranial FUS applications outside of the MRI system without a stereotactic frame, thereby benefiting patients especially in the elderly population.

Toward a Cognitive Neural Prosthesis Using Focused Ultrasound.

Non-invasive brain stimulation using focused ultrasound has many potential applications as a research and clinical tool, including its incorporation as either an extracorporeal or implantable neural prosthetic. To this end, we investigated the effect of focused ultrasound (FUS) combined with systemically administered microbubbles on visual-motor decision-making behavior in monkeys. We applied FUS to the putamen in one hemisphere to open the blood-brain barrier (BBB), and then tested behavioral performance 3-4 h later. On days when the monkeys were treated with FUS, their decisions were faster and more accurate than days without sonication. The performance improvement suggested both a shift in the decision criterion and an enhancement of the use of sensory evidence in the decision process. FUS also interacted with the effect of a low dose of haloperidol. The findings indicate that a two-minute application of FUS can have a sustained impact on performance of complex cognitive tasks, and may increase the efficacy of psychoactive medications. The results lend further support to the idea that the dorsal striatum plays an integral role in evidence- and reward-based decision-making, and provide motivation for incorporating FUS into cognitive neural prosthetic devices.

Inferential Learning of Serial Order of Perceptual Categories by Rhesus Monkeys (Macaca mulatta).

Category learning in animals is typically trained explicitly, in most instances by varying the exemplars of a single category in a matching-to-sample task. Here, we show that male rhesus macaques can learn categories by a transitive inference paradigm in which novel exemplars of five categories were presented throughout training. Instead of requiring decisions about a constant set of repetitively presented stimuli, we studied the macaque's ability to determine the relative order of multiple exemplars of particular stimuli that were rarely repeated. Ordinal decisions generalized both to novel stimuli and, as a consequence, to novel pairings. Thus, we showed that rhesus monkeys could learn to categorize on the basis of implied ordinal position, without prior matching-to-sample training, and that they could then make inferences about category order. Our results challenge the plausibility of association models of category learning and broaden the scope of the transitive inference paradigm.SIGNIFICANCE STATEMENT The cognitive abilities of nonhuman animals are of enduring interest to scientists and the general public because they blur the dividing line between human and nonhuman intelligence. Categorization and sequence learning are highly abstract cognitive abilities each in their own right. This study is the first to provide evidence that visual categories can be ordered serially by macaque monkeys using a behavioral paradigm that provides no explicit feedback about category or serial order. These results strongly challenge accounts of learning based on stimulus-response associations.

Targeting Effects on the Volume of the Focused Ultrasound-Induced Blood-Brain Barrier Opening in Nonhuman Primates In Vivo.

Drug delivery to subcortical regions is susceptible to the blood-brain barrier (BBB) impeding the molecular exchange between the blood stream and the brain parenchyma. Focused ultrasound (FUS) coupled with the administration of microbubbles has been proved to open the BBB locally, transiently, and noninvasively both in rodents and in nonhuman-primates (NHPs). The development of this disruption technique independent of MRI monitoring is of primordial importance yet restrained to the targeting optimization. This paper establishes the linear relationship of the incidence angle with the volume of BBB opening ( VBBB ) and the peak negative pressure when sonicating the caudate nucleus and the putamen region of five NHPs. In addition, the effect of central nervous system structures on the opening morphology is evaluated by identification of the gray-to-white-matter ratio at the opening site. Finally, the targeting accuracy is assessed through the estimation of the geometric and angle shift of the opening from the targeted region. Interestingly, results prove a monotonic increase of the opening volume with close to normal incidence angles. Moreover, 80.35% of the opening lies on gray-matter regions compared with only 19.41% attributed to the white matter. The opening was found to be shifted axially, toward the transducer, and laterally with an average angle shift of 4.5°. Finally, we were capable of showing reproducibility of targeting accuracy with the same stereotactic and ultrasonic parameters. This paper documents the a priori prediction of the opening volume through manipulation of the angle and pressure as well as establishing the predictability, accuracy, and safety of FUS-induced BBB opening in NHPs.